Timing-accurate Full-system Multiprocessor Simulations Can Take Years Because of Architecture and Application Complexity. Statistical Sampling Makes Simulation-based Studies Feasible by Providing Ten-thousand-fold Reductions in Simulation Runtime and Enabling Thousand-way Simulation Parallelism. Simflex: Statistical Sampling of Computer System Simulation

Simulating a single CPU is typically thousands of times slower than the actual CPU. Full-system multiprocessor simulation, which involves simulating many CPUs, peripherals, and other system components on a single host, compounds the slowdown by another factor of 10 to 100. In other words, multiprocessor simulation is up to a million times slower than real hardware. This speed difference leads to prohibitively long turnaround times for simulating complete computer benchmarks—in particular, multiprocessor server benchmarks. (These benchmarks are often longer than their uniprocessor counterparts to compensate for nondeterministic thread scheduling and for perturbation effects from the operating system and I/O that can lead to significant shortterm performance variations.) Statistical sampling makes full-system multiprocessor simulation feasible by reducing simulation times by roughly a factor of 10,000. Sampling provides such drastic reductions by exploiting the homogeneity of application performance—application behaviors that repeat millions of times. By applying rigorous statistical methods, we can identify the minimal sample that assesses application performance with a desired confidence level. Our first work on this topic, Smarts, investigated statistical sampling of the System Performance Evaluation Cooperative (SPEC) CPU2000 benchmarks on uniprocessor simulators. That work demonstrated that the nature of performance variability across measurement granularities favors a large sample of thousands of brief execution windows to minimize total simulation. The primary challenge in realizing sampling’s drastic acceleration lies in rapidly constructing the correct initial state for the large number of fine-grained performance measurements; we call this the warming problem. Our first solution to this problem uses a simplified simulation model to maintain architectural and selected microarchitectural state while fast-forwarding between measurements. Although this approach provides accurate results, it does not realize the full potential of statistical sampling: The time spent fastforwarding (making up 99 percent of experiment turnaround time) grows with Thomas F. Wenisch Roland E. Wunderlich Michael Ferdman Anastassia Ailamaki Babak Falsafi James C. Hoe